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An  editor in  J  in  Journal  of  Marketing  Kesi-arch  ten  years  ago  warned  us 
of  the  pitfalls  of  incorrect  research  procedures.   That  editorial  is  updated, 
with  special  emphasis  on  research  conducted  in  the  laboratory. 


It  has  been  ten  years  since  Bob  Fcrber  told  us  "How  Not  to  Do  Research" 
(1).   As  Ferber  put  it: 

How-to  publications  are  popular  today,  especi.'-lly  with 
the  reputed  publication  of  'Brain  Surgery  Self-Taught.' 
Though  it  isn't-  likely  that,  anything  as  sophisi  iented 
will  happen  in  marketing  research,  marketing  researchers 
should  try  to  prepare  material  of  i.-.ju.YJ  value  for  their 
profession.   Any  marketing  researcher,  no  matter  what  his 
experience  or  sophistication,  with  liLile  effort  can  get 
himself  relieved  of  arduou.-;  research. 

In  the  ten  year  interim,  we  have  again  been  besieged  with  how-to  publications. 
It  is  time  for  a  refresher  course. 

My  own  research  to  date  has  been  concentrated  in  the  laboratory,  so  there 
I  will  focus.   I  feel  particuarly  qualified  in  relating  how  not  to  do  research 
in  the  laboratory,  not  because  I  have  more  experience  in  the  lab  than  others, 
but  because  I  have  more  experience  doing  the  wrong  thing  in  the  lab  than 
others. 

The  following  list  of  ten  rules  includes  the  more  important  aspects  of 
how  not  to  conduct  lab  research.   It  is  not  intended  to  be  exhaustive.   Nor 
is  it  presented  so  that  others  might  profit  from  my  mistakes.   Indeed,  in 
this  dog-eat-dog  profession  of  ours,  I  encourage  those  researchers  nipping 
at  the  heels  of  my  academic  security  to  find  out  for  themselves  the  conse- 
quences of  following  in  my  footsteps.   And  if  along  the  pall),  additional  ways 
are  discovered  not  to  do  research,  please  submit  them  to  the  author  who  will 
file  them  for  the  next  update  in  3988: 

1.  Never  base  lab  studies  on  theory.  Remember  that  theories  can  never 
be  proved,  only  disproved.   So  why  take  a  chance?   If  your  religious 
convictions  compel  you  to  make  hypotheses,  at  least  have  the  good 
sense  to  make  them  after  you  conduct  the  study  rather  than  before. 


You  will  then  have  the  aid  of  a  hundred  or  more  regressions,  anovas, 
and  crosstabs  of  which  at  least  five  percem  will  surely  be 
significant. 

2.  Always  aim  for  at  least  four  independent  variable.-:  with  three  or 
more  levels  of  each.   Do  not  worry  about  replications—empty  cells 
are  rarely  a  problem.   Besides,  main  effects  -ire  no  longer  of  in- 
terest, and  most  one  and  two  way  interactions  have  already  been 
established.   Only  three  and  four  way  interactions  remain  as 
palatable  food  for  the  serious  researcher.  • 

3.  Do  not  use  multiple  measures.  Why  burden  your  study  with  redun- 
dancy? What  would  Copernicus  say  about  multi-trait,  multi-method 
as  a  philosophy  of  science?   Surely  that  it  violates  the  law  of 
parsimony.   It  is  time  to  put  uni- trait,  uni-raethod  back  into  the 
lab.   What's  true  measured  one  way  is  true  measured  another.   If 
you've  seen  one  measure,  you've  seen  them  all...   This  would  also 
eliminate  the  need  for  any  sophisticated  analysis.   Simply  follow 
the  four  C's:  Collect  it.   Code  it.   Cross-tab  it.   Communicate  it. 

4.  Continue  to  use  college  sophomores  as  subjects  for  all  studies. 
They  are  available,  cheap,  and  well-healed.   Having  been  in  the  lab 
many  times  before,  instructions  are  rarely  necessary.   They  know 
what  behaviors  are  expected  of  thorn  and  will  cause  you  no  trouble. 
You  need  not  disguise  the.  true  purpose  of  the  experiment — they  will 
know  it  anyway — even  if  you  don't.   Demand  characteristics  exist 
only  in  the  chapters  of  textbooks  and  the  lectures  of  professors 
seeking  ways  to  fill  their  courses  with  anything  but  substance. 
Besides,  external  validity  is  no  longer  in  vogue. 


You  will  then  have  the  aid  of  a  hundred  or  more  regressions,  anovas, 
and  crosstabs  of  which  at  lease  five  percent  will  surely  be 
significant. 

2.  Always  aim  for  al:  least  four  independent  variables  with  three  or 
more  levels  of  each.   Do  not  worry  about  replicaLions--empty  cells 
are  rarely  a  problem.   Besides,  main  effects  are  no  longer  of  in- 
terest and  most  one  and  two  way  interactions  have  already  been 
established.   Only  three  and  four  way  interactions  remain  as 
palatable  food  for  the  serious  researcher.  ■ 

3.  Do  not  use  multiple  measures.  Why  burden  your  study  with  redun- 
dancy? What  would  Copernicus  say  about  multi-trait,  multi-method 
as  a  philosophy  of  science?  Surely  that  it  violates  the  law  of 
parsimony.   It  is  time  to  put  uni -trait,  uni-iaethod  back  into  the 
lab.  What's  true  measured  one  way  is  true  measured  another.   If 
you've  seen  one  measure,  you've  seen  them  all. . .   This  would  also 
eliminate  the  need  for  any  sophisticated  analysis.   Simply  follow 
the  four  C's:  Collect  it.   Code  it.   Cross-tab  it.   Communicate  it. 

4.  Continue  to  use  college  sophomores  as  subjects  for  all  studies. 
They  are  available,  cheap,  and  well-healed,   Having  been  in  the  lab 
many  times  before,  instructions  are  rarely  necessary.   They  know 
what  behaviors  are  expected  of  them  and  will  cause  you  no  trouble. 
You  need  not  disguise  the  true  purpose  of  the  experiment — they  will 
know  it  anyway — even  if  you  don't.   Demand  characteristics  exist 
only  in  the  chapters  of  textbooks  and  the  lectures  of  professors 
seeking  ways  to  fill  their  courses  with  anything  but  substance. 
Besides,  external  validity  is  no  longer  in  vogue. 


5.  lie  sure  that  experimenters  are  fully  aware  of  expected  outcomes  and 
have  a  vested  interest  in  the  success  of  the  experiment,   This  might 
be  achieved  by  compensating  them  on  n   commission  basis  (e.g.,  so 
much  per  successful  session)  or  offering  co-authorship  status  in 
the  resulting  publication  if  the  experiment  works. 

6.  Pre-test  minimally,  if  at  all,  and  only  because  your  now-ancient 
dissertation  advisor  showed  you  his  stone  tablet  with  the  inscription 
"Thou  shalt  pre-test"  when  be  agreed  to  take  you  on  as  his  disciple. 
What  can  possibly  go  wrong  that  you  haven't  already  thought:  of?   If 
you  feel  you  must  pre-test,  use  only  friends,  relatives,  or  colleague.' 
so  you  don't  have  to  pay  them.   Simply  tell  them  to  act  like  your 
criterion  population.   Better  yet,  toll  them  to  act  like  college 
sophomores.   And  don't  forget  to  include  pre-tesr  results  as  part 

■  of  the  final  analysis — an  excellent:  way  to  increase  sample  size. 

7.  Make  sure  that  your  post-test  questionnaire  is  comprehensive.   As 
long  as  you  have  a  captive  audience,  why  not  make  the  most  of  it? 
Begin  by  writing  down  in  the  form  of  a  question,  every  item  that 
could  conceivably  relate  to  your  experiment,   include  all  these 
items  on  the  final  questionnaire.   Be  sure  not  uo  restrict  yourself 
to  items  you  know  how  to  analyze.   Such  icems  make  super!)  covariates. 
Now  make  up  another  set  of  items  that  relate  to  any  other  research 
you  are  also  engaged  in.   Finally,  solicit  additional  items  from 
friends  or  colleagues  who  arc  in  need  of  data — they  will  feel 
obliged  to  do  the  same  for  you  some  day. 

8.  Throw  out  bad  data  as  you  go  along.   You  are  in  the  lab  because  of 
its  unique  capacity  to  permit  strict  control,  so  be  sure  to  take 
advantage  of  that  capacity.   If  something  unusual  or  unanticipated 


occurs,  it  is  surely  outside  the  boundaries  you  have  established 
for  the  conduct  of  the  study.   Simply  discard  it  and  run  an  extra 
session.   If  you  don't,  somebody  else  will  explain  the  results 
that  you  couldn't.   Why  subject  yourself  to  such  embarrassment? 
9.   Don't  bother  to  debrief  your  subjects.   This  is  simply  a  waste  of: 
time  and  might  even  result  in  bringing  abuse  upon  yourself  when 
subjects  find  they  have  been  deceived  once  again.   If  you  don't 
debrief,  who  will  ever  know?  Certainly  not  the  Committee  on  Human 
Subjects  to  whom  you  swore  you  would. 
10.   Finally,  never  design  a  study  which  is  complete  in  and  of  itself. 
As  academics,  pledged  to  combat  anything  bordering  on  closure,  we 
live  in  constant  fear  of  one  day  reading  the  journal  article  that 
concludes,  "Further  research  in  this  area  will  not  be  necessary." 


Peter  H.  Webb  is  Assistant  Professor  of  Business  Administration  at  the 
University  of  Illinois.  Preparation  of  this  article  was  aided  by  the 
grimaces  and  guffaws  of  Professor  Vithala  Rao,  whom  the  author  wishes  to 
thank  for  his  assistance. 
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